Combine all tests into a single suite to increase parallelism. NFC#26365
Merged
sbc100 merged 1 commit intoemscripten-core:mainfrom Mar 4, 2026
Merged
Combine all tests into a single suite to increase parallelism. NFC#26365sbc100 merged 1 commit intoemscripten-core:mainfrom
sbc100 merged 1 commit intoemscripten-core:mainfrom
Conversation
kripken
reviewed
Feb 28, 2026
sbc100
changed the title
This allows all tests to be run in parallel. For example:
```
Test suites: ['test_core', 'test_jslib', 'test_other']
Running test_core: (88 tests)
Using 88 parallel test processes
[88/88] test_asan_vector (test_core.core0.test_asan_vector) ... ok
----------------------------------------------------------------------
Ran 88 tests in 3.296s
OK (skipped=5)
Running test_jslib: (54 tests)
Using 54 parallel test processes
[54/54] test_jslib_aliases_closure_wasm64 (test_jslib.jslib.test_jslib_aliases_closure_wasm64) ... ok
----------------------------------------------------------------------
Ran 54 tests in 5.287s
OK
Running test_other: (41 tests)
Using 41 parallel test processes
[41/41] test_abspaths (test_other.other.test_abspaths) ... ok
----------------------------------------------------------------------
Ran 41 tests in 5.210s
OK
Total core time: 228.881s. Wallclock time: 13.795s. Parallelization: 16.59x.
====================
TEST SUMMARY
test_core: 88 run, 0 errors, 0 failures, 5 skipped
test_jslib: 54 run, 0 errors, 0 failures, 0 skipped
test_other: 41 run, 0 errors, 0 failures, 0 skipped
```
After:
```
$ ./test/runner jslib other.test_a* core0.test_a* --skip-slow
Running 183 tests
Using 128 parallel test processes
[183/183] test_abspaths (test_other.other.test_abspaths) ... ok
----------------------------------------------------------------------
Ran 183 tests in 7.459s
OK (skipped=5)
Total core time: 301.490s. Wallclock time: 7.459s. Parallelization: 40.42x.
```
Note the wall clock time is less since we ran all the tests in parallel.
If you try to mix parallel and non-parallel test module you now get an
error. e.g.:
```
$ ./test/runner core0 benchmark
runner: error: attempt to mix parallel and non-parallel test modules
```
kripken
approved these changes
Mar 4, 2026
| # some native-dependent tests fail because of the lack of native | ||
| # headers on emsdk-bundled clang | ||
| test_targets: "other jslib skip:other.test_native_link_error_message" | ||
| - run-tests-linux: |
Member
There was a problem hiding this comment.
Can you duplicate steps like this? I wasn't aware of that
Collaborator
|
This should be non-breaking, i.e. none of the existing suite names change in any way? |
juj
approved these changes
Mar 4, 2026
Collaborator
juj
left a comment
juj
left a comment
There was a problem hiding this comment.
LGTM - this is a nice speedup.
Collaborator
Author
Yes, its non-breaking. The only difference is that if you run multiple suites at once then all the tests that run and summarized together. We no longer give per-module/per-suite summary information at the end, but this is also how python tests normally work IIUC. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
My motivation for this change is to allow for increased parallelism that is not limited to how
many tests are in a give module. I'm hoping to do more splitting up of test_other.py, for
example, and I don't want to loose parallism.
After this change all tests are combined into a single parallel test suite. For example:
After:
Note the wall clock time is less since we ran all the tests in parallel.
If you try to mix parallel and non-parallel test module you now get an error. e.g.: